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In recent years, policymakers’ and practitioners’ interest in school climate as a contributor to K-12 student 
learning and classroom processes has increased, both in the US and internationally. However, researchers 
have not yet examined the influence of school climate on the youngest learners in these contexts — 
prekindergartners. Using data from the Boston Public Schools, where the public prekindergarten pro- 
gram is housed in elementary schools, we explored associations among dimensions of school climate 
as reported by teachers and older peers, observed measures of prekindergarten classroom quality, and 
gains in children’s receptive vocabulary and executive function across the prekindergarten year. Student 
participants included 299 children attending full-day prekindergarten programs across 35 elementary 
schools in 61 classrooms. Findings indicated that school emotional climate had small positive association 
with prekindergarten classroom emotional support. School-level measures of teacher effectiveness also 
had positive associations with prekindergarten classroom levels of emotional support and classroom 
organization. School climate dimensions were not associated with children’s receptive vocabulary and 
executive function gains across the prekindergarten year. Results are discussed in the context of current 


school climate research and state-wide accountability efforts to assess school quality. 


© 2019 Elsevier Inc. All rights reserved. 


1. Introduction 


Interest in school climate as a contributor to K-12 student 
learning and classroom processes has been on the rise in the 
U.S. and internationally (Berkowitz, Moore, Astor, & Benbenishty, 
2017; Thapa, Cohen, Guffey, & Higgins-D’Alessandro, 2013; U.S. 
Department of Education, 2007). Research has consistently linked 
a more positive school organizational climate with a host of ben- 
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efits for students and teachers, including higher student academic 
achievement (e.g., Cornell, Shukla, & Konold, 2016; Sherblom, 
Marshall, & Sherblom, 2006) and less teacher turnover (Kraft, 
Marinell, & Yee, 2016). Motivated by these findings, school climate 
has increasingly become a focus of policymakers. The Every Stu- 
dent Succeeds Act (ESSA) recently mandated states to include a 
non-academic indicator of school quality across all schools, with 
school climate measures as one of the options for doing so (U.S. 
Department of Education, 2015). As of 2016, six out of 17 states 
that had submitted ESSA plans had chosen school climate as one of 
their key ESSA indicators (American Institutes for Research, 2016). 

School climate indicators also have potential implications for 
public prekindergarten classrooms. In recent years, states and local- 
ities have expanded access to publicly funded prekindergarten 
programs and about half of all state- and locally-funded prekinder- 
garten seats are in public schools (Barnett, Carolan, Fitzgerald, & 
Squires, 2012; Barnett, Carolan, Squires, & Clarke Brown, 2015). 
However, there is little research on the influence of school cli- 
mate on prekindergarten classrooms and student outcomes. In 
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the present exploratory study, we add to the school climate lit- 
erature by examining associations between school climate and 
three domains of prekindergarten classroom quality (emotional 
support, classroom organization, instructional quality) within a 
diverse sample of prekindergarten children attending a full-time 
program in the Boston Public Schools (BPS). We also consider the 
role of school climate in supporting growth in children’s receptive 
vocabulary and executive functioning — two cognitive skills that are 
important components of children’s school readiness (Blair & Razza, 
2007) - across the prekindergarten year. Given the growing atten- 
tion that policymakers are paying to school climate in elementary 
schools, our results help fill important gaps in current understand- 
ing of the contexts that best support the classroom experiences and 
academic outcomes of our youngest learners. 


1.1. Defining and measuring school climate and school climate 
dimensions 


Research recognizes that a school’s climate is comprised of a 
complex set of interrelated elements (Cohen, McCabe, Michelli, & 
Pickeral, 2009; Wang & Degol, 2016). The National School Climate 
Committee (NSCC) defines school climate as the “quality and char- 
acter of school life” (NSCC, 2007) and asserts that school climate 
is comprised of the patterns of values, norms, and goals that are 
shared among a school community, including staff, teachers, stu- 
dents, families, and the broader community. Although there is no 
consensus regarding the specific dimensions of school climate, four 
central areas recur in empirical reviews: relationships (e.g., how 
connected individuals in school community feel to one another), 
teaching and learning (e.g., values and norms that promote student 
academic achievement), safety (e.g., social, emotional, intellectual, 
and physical security), and institutional environment (e.g., school 
curricular, interpersonal, and physical resources; Berkowitz et al., 
2017; Thapa et al., 2013). 

The University of Chicago’s Consortium of School Research 
(CCSR) developed the most comprehensive and empirically tested 
school climate framework that addresses all four areas (see 
columns 1-4, Table 1; Bryk, Sebring, Allensworth, Luppescu, & 
Easton, 2010). Informed by decades of descriptive reviews of 
Chicago school reform efforts (Bryk, Sebring, Kerbow, Rollow, & 
Easton, 1998; Designs for Change, 1993), the Essential Supports 
of School Improvement framework (referred to as the Essen- 
tial Supports framework) identifies several related but distinct 
school organizational elements that are most likely to support 
improvements in student learning: a school leadership element that 
represents the school’s instructional support and how principals 
engage teachers and staff in school decision making processes; 
a professional capacity element that includes quality collabora- 
tions among teachers and teacher attitudes toward teaching and 
learning; a parent-community ties element that includes teachers 
learning about local culture and parent involvement; a student- 
centered learning climate element that addresses school safety as 
well as student support and engagement; and an instructional guid- 
ance element wherein schools focus on curricular demands and 
approaches to learning. 

A strength of the Essential Supports framework is that the CCSR 
has examined the framework for individual- and school-level reli- 
ability, item difficulty, and item infit using Rasch modeling and 
has found it has good psychometric properties (Bryk, Sebring, 
Allensworth, Luppescu, & Easton, 2009, 2010; Luppescu & Ehrlich, 
2012). The framework is currently implemented widely in the 
Chicago Public Schools and has been adopted by the NSCC and other 
large-scale school districts across the country (e.g., New York City). 
To date, this framework has not been tested in prekindergarten — 
a gap our study addresses. 


1.2. Theoretical foundation for the influence of school climate on 
prekindergarten classroom quality and child outcomes 


There are reasons to think that the Essential Supports frame- 
work could work equally as well in prekindergarten as it appears 
to in the older grades, for many of the same reasons. In their con- 
ceptualization of how the Essential Supports influence children’s 
classroom and learning experiences, Bryk et al. (2010) theorize 
that the Essential Supports directly inform the dynamics of student 
learning, such as student motivation and school participation, and 
technical aspects of classroom learning, such as classroom instruc- 
tion and the effective use of learning time. Thus, according to the 
Essential Supports framework, school climate should affect class- 
room quality and students’ cognitive outcomes directly via the 
interactions teachers and students have within the classroom and 
broader school contexts. 

The quality of teacher-student interactions is particularly 
important for prekindergarten children who, like some of their 
elementary school peers, spend most of their day embedded in 
their classroom context but whose classroom quality and aca- 
demic gains could still be influenced by processes at the school 
level. For example, as related to children’s classroom quality, 
prekindergarten students in schools with the capacity to provide 
effective instructional guidance (instructional guidance element in 
the Essential Supports framework) and retain effective teachers 
can benefit from teachers’ collective knowledge and skills, which 
can directly influence classroom instructional quality. In relation 
to children’s academic gains, schools that value input from parents 
and the broader community may enhance parent-teacher trust 
(parent-community ties essential) which could directly improve 
student-teacher classroom relationships (i.e., joint problem solv- 
ing around academic issues) and directly influence child outcomes 
as parents create routines at home to support children’s learning 
at school. Another possible direct pathway is one in which chil- 
dren who feel safe in school (the student-centered learning climate 
essential) are less distracted and can focus more attention on get- 
ting acclimated to classroom routines, such as transitioning from 
one classroom activity to the next and communicating with teach- 
ers and peers. 

Prekindergarten is also a distinctive context in several respects. 
In prekindergarten, there tends to be particular emphasis on devel- 
oping foundational skills, such as building a rich vocabulary to 
support subsequent academic learning, listening to and follow- 
ing spoken directions, and focusing one’s attention. It is unclear if 
current measures of school climate are likely to capture the contex- 
tual supports that are particularly important for prekindergarten 
children, such as nurturing and individualized support for early 
language and cognitive development (Ehrlich, Pacchiano, Stein, & 
Luppescu, 2016). In addition, standards for prekindergarten chil- 
dren (e.g., language/literacy and approaches to learning standards 
in prekindergarten; Massachusetts Department of Early Education 
and Care, 2005; Michigan State Board of Education, 2005; New 
York State Education Department, 2011; Oklahoma Department of 
Education, n.d.) are different than those for older children (e.g., state 
standards for reading, writing, listening and speaking; National 
Governors Association Center for Best Practices & Council of Chief 
State School Officers, 2010; Michigan Department of Education, 
2017) and existing school climate measures may or may not be 
sensitive to these differences. 

Regarding our focal child outcomes specifically - vocabulary 
and executive function development — schools with higher teacher 
professional capacity (e.g., one of the Essential Supports) might 
have prekindergarten teachers who employ multiple approaches 
(e.g., providing both oral and visual directions) when helping chil- 
dren who may need additional support apply new knowledge to 
different classroom activities and when building children’s vocab- 
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Table 1 


Areas and features of school climate research, the Essential Supports framework for school improvement (Bryk et al., 2010), and study school climate dimensions. 


Central areas of school climate Defining features 


Related Essential Support(s) 


Sample Essential Support BPS school climate dimensions 


indicators 


Relationships Adult social support, peer 
social support, teacher’s work 


environment 


Teaching and learning Support for learning (e.g., 
academic challenge), social and 


civic learning 


climate 


Safety Rules and norms, physical 
security, social-emotional climate 
security 

Institutional environment School 


connectedness-engagement, 
physical school layout and 
surroundings, resources and 
supplies 


School leadership, faculty and 
professional capacity 


Student-centered learning 


Student-centered learning 


Parent and community ties, 
instructional guidance 


Instructional leadership, 
inclusive facilitative 
leadership, collective 
responsibility, school 
commitment 

Academic engagement, peer 
support for academic work, 
press toward academic 
achievement from teachers 
Safety and order 


Instructional leadership, 
principal effectiveness, work 
environment collegiality, 
perceptions of collective 
efficacy 

Enthusiasm for learning, 
teacher effectiveness, teacher 
self-efficacy, teacher influence 


Positive emotional climate 


Parent and student 
engagement 


Use of community resources, 
parent involvement in school, 
curriculum content and 
academic demands 


ulary knowledge. School climate might also operate differently on 
prekindergarten children’s executive function (EF) gains compared 
to older students, given that EF skills develop particularly rapidly 
during the preschool period (Zelazo, Muller, Frye, & Marcovitch, 
2003) and are particularly important when children first enter 
formal schooling and are tasked with remembering rules and direc- 
tions, sitting for longer periods of time, and inhibiting prepotent 
responses to play with materials or speak when the teacher is 
speaking. Classroom processes, such as effective classroom man- 
agement, are important in fostering children’s EF development 
(Ursache, Blair, & Raver, 2012), yet little is known how school pro- 
cesses also support these skills. 


1.3. Empirical evidence linking school climate to classroom 
quality and child outcomes 


Empirically, there has been little work examining the central 
school climate areas of teaching and learning, safety, and insti- 
tutional environment - e.g., the influence(s) of these areas on 
classroom-level processes —- in any grade. At the prekindergarten 
level, there has been some work linking school climate-like con- 
structs related to organizational relationships, such as teacher 
collegiality and supervisor support, to classroom quality (e.g., 
Dennis & O'Connor, 2013; Zinsser & Curby, 2014), though none 
of these studies was based explicitly on the Essential Supports 
framework. So far, prekindergarten findings regarding relational 
aspects of school climate have been mixed, with some studies 
showing positive associations between school climate and global 
measures of classroom quality (Dennis & O’Connor, 2013; Lower & 
Cassidy, 2007) and some showing no or inconsistent associations 
between school climate and measures of classroom quality focused 
on teacher-child interactions, like emotional support (McGinty, 
Justice, & Rimm-Kaufman, 2008; Zinsser & Curby, 2014). Further, it 
is not clear that this prior work generalizes to prekindergarten pro- 
grams based in public schools; most prior studies only examined 
community-based preschools, which do not generally include stu- 
dents in kindergarten and later grades, and tend to have a different 
structure and set of inputs than public schools (e.g., lower pay, no 
union-protected common planning time, more flexible arrival and 
departure schedules for students; Yudron, Weiland, & Sachs, 2018). 

There has been far more evidence linking school climate with 
student academic outcomes within elementary, middle, and high 
school samples (though primarily, at the high school level). While 
no empirical work has investigated the link between school cli- 
mate and students’ vocabulary knowledge or executive function 
skills, some work has examined the influence of school climate 


on related outcomes such as general measures of language/literacy 
(e.g., rhyming, reading fluency) and standardized reading/English 
language arts scores. Work exploring the influence of school climate 
on child-level processes has been mixed. While research has gen- 
erally found associations between supportive school climate and 
student academic outcomes from elementary school through high 
school (e.g., Berkowitz et al., 2017; Bryk et al., 2010; Nathanson 
et al., 2013; Thapa et al., 2013), some studies that include elemen- 
tary school climate have found little correlation between climate 
and gains in student academic skills (e.g., Scott, Parsley, & Fantz, 
2014). We are aware of just one relevant study that has found 
positive associations of adult support for academic success and 
school safety specifically for kindergarten children (Lowenstein, 
Friedman-Krauss, Raver, Jones, & Pess, 2015). In this study, the 
authors investigated the influence of sixth through eighth grade 
student-reports of adult support and safety (termed unsafe cli- 
mate) on kindergarten children’s academic skills and found that 
children in schools categorized as having low adult support were 
rated by their teachers as having poorer language/literacy and math 
skills (Lowenstein et al., 2015). This association was moderated 
by student-teacher relationships at the classroom level. School 
safety, however, was not related to kindergarten children’s aca- 
demic skills. 

In sum, to date, there have been few empirical tests of whether 
dimensions of school climate influence classroom quality and of 
whether prekindergarten children make greater cognitive skill 
gains in public schools with a more positive school climate. Fur- 
ther, we are aware of no studies to date that have examined the 
associations between school climate and both classroom quality 
and cognitive gains — specifically receptive vocabulary and execu- 
tive function, two constructs that are predictive of later academic 
development (Duncan et al., 2007) — during the prekindergarten 
year. More empirical evidence is needed, given that many young 
students begin their public school careers in prekindergarten and 
given the broader policy focus on school climate. 


1.4. The present study 


In the present study, we add to the school climate and prekinder- 
garten literature by examining the relationship between school 
climate (as perceived by teachers and students in grades 3-8), 
observed prekindergarten classroom quality, and prekindergarten 
children’s cognitive gains in programs located in public schools. We 
also sought to determine the reliability and validity of the school 
climate dimensions captured by district-wide surveys. We explored 
two specific research questions: 
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Table 2 
Classroom-level teacher experience and quality and school-level covariates. 
Characteristic Mean or proportion (SD) Min Max 
Classroom-level teacher experience and quality (N=61 classrooms) 
Teacher had a master’s degree 0.82 (0) 1 
Teacher had 3-5 years of experience 0.09 0 1 
Teacher had 5-10 years of experience 0.27 0 1 
Teacher had 10+ years of experience 0.56 0 1 
Spring CLASS emotional support score 5.63 (0.69) 3 6.63 
Spring CLASS classroom organization score 5.14 (0.71) 2.75 6.07 
Spring CLASS instructional support score 4.34 (0.93) 1.92 5.67 
School-level covariates (N= 35 schools) 
% of 3rd graders scoring at or proficient 35.09 (17.66) 7.00 73.00 
% of low income students 77.34 (13.95) 35.40 94.20 
% of students with disabilities 17.97 (5.34) 10.80 37.70 
% of students who are English language learners 22.10 (14.70) 2.40 53.10 
% of teachers retained 85.08 (10.35) 51.80 100 
Stability rate 83.73 (6.66) 62.40 95.50 
Student/teacher ratio 13.67 (1.68) 9.20 17.50 


Note: Stability rate is calculated as the percent of student who remain in the district/school from one school year to the next. At the classroom level, teacher education was 
missing for 2% of the sample and teacher years of experience were missing for 10% of the sample. All other data are non-missing. 


1) Does a more supportive school climate as reported by teach- 
ers and older children (grades 3-8) predict higher observed 
prekindergarten classroom quality? 

2) Does a more supportive school climate predict gains in 
prekindergarten children’s receptive vocabulary and EF skills? 


2. Method 
2.1. Participants and setting 


All children sampled in the present study were enrolled in 
the Boston Public School (BPS) full-day prekindergarten program 
(6.5 h) during the 2009-2010 academic year. The program was open 
to all four-year-old children living in the city of Boston, regardless 
of income. District guidelines required class size to be capped at 22, 
and classrooms to be staffed by a teacher and a paraprofessional, 
for a maximum teacher-student ratio of 1:11. All BPS prekinder- 
garten teachers were subject to the same educational requirements 
and pay scale as K-12 teachers. Prekindergarten teachers also were 
required to have an early childhood (pre-K to grade 2) license from 
the Massachusetts Department of Elementary and Secondary Edu- 
cation (Jason Sachs, personal communication, December 3, 2018). 
As shown in Table 2, 82% of the teachers in the sample had a master’s 
degree. 

In spring 2010, 77 Boston public elementary schools admin- 
istered the district’s school climate survey (see Fig. 1). Of those 
schools, 62 served prekindergarten children. Five schools were 
excluded from the eligible study sample because they did not 
include students who completed the school climate survey (e.g., 
children in third grade or higher) and another three did not have 
complete teacher and student climate data, leaving 54 schools eli- 
gible schools for our sample. Of these, 35 had fall 2009 and spring 
2010 child assessment data available, due to their participation 
in studies of the program’s impact, and had non-missing aver- 
age school-level achievement information available, a covariate 
included in our model (described later) that may influence the aca- 
demic achievement of young children (e.g., Zhai, Raver, & Jones, 
2012). Our final sample included 299 prekindergarten children 
from 61 classrooms across 35 schools that had prekindergarten 
programs and school climate data. 

As shown in Table 3, focal children were diverse in their back- 
ground characteristics and were approximately representative of 
the district’s prekindergarten population. On average, 77% of stu- 
dents attending sample schools qualified for free- or reduced-price 
lunch and 46% of students were Hispanic. Children attended either 


Boston Public Schools (BPS} elementary schools 
that administered 2009-2010 climate survey 
N=77 


BPS elementary schools with pre-K 
that administered climate survey 
N=62 


BPS elementary schools with pre-K and climate survey 
that had 3" grade children 
N=57 


BPS elementary schools with pre-K and climate survey 


that had 3° grade children and complete climate data 
N=54 


BPS elementary schools with pre-K and climate survey 
that had 3" grade children and complete climate data, 
and who participated in Fall 2009 and 
Spring 2010 data collection 
N=36 


BPS elementary schools with pre-K and climate survey 
that had 3" grade children and complete climate data, 
who participated in Fall 2009 and 
Spring 2010 data collection, 
and had school-level MCAS scores 
N=35 


Fig. 1. Diagram of final sample of schools from Boston prekindergarten program 
and climate survey administration. 
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Table 3 
Child- and classroom-level sociodemographic characteristics. 


Child-level (N=299) 


Classroom-level (N=61) 


Characteristic Mean or proportion (SD) 


Fall PPVT scores 45.30 (20.16) 
Spring PPVT scores 60.36 (21.32) 
Fall EF scores 0.49 (0.22) 
Spring EF scores 0.60 (0.19) 
Mobile 0.12 
Race/ethnicity: Asian 0.13 
Race/ethnicity: Black 0.28 
Race/ethnicity: Hispanic 0.40 
Race/ethnicity: Other 0.02 

Male 0.52 

Home language is Spanish 0.27 

Home language is not Spanish or English 0.23 

Eligible for free/reduced lunch 0.71 

Special needs 0.12 

East attendance zone 0.55 

North attendance zone 0.21 


Min Max Proportion (SD) Min Max 
3 105 - - - 
9 118 - - - 
0.10 0.94 - - - 
0.14 0.92 - - - 
0 1 0.22 0 0.77 
0 1 0.10 0 1 
0 1 0.29 0 1 
0 1 0.41 0 1 
0 1 0.02 0 0.18 
0 1 0.48 0 0.75 
0 1 0.24 0 1 
0 1 0.20 0 1 
0 1 0.70 0.05 1 
0 1 0.12 0 0.50 
0 1 0.50 0 1 
0 1 0.22 0 1 


Note: N=299 children in 61 classrooms in 35 schools. At the child level, fall PPVT was missing for 6% of the sample and fall EF was missing for 9% of the sample. All other data 


are non-missing. 


P-5 (N=26) or P-8 schools (N=9). We found two statistically sig- 
nificant differences between the sample population of schools 
included in the study and all other district schools serving 4- 
year-old children across eight available school-level background 
characteristics: percent Black (34% participating vs. 36% not partic- 
ipating; p<0.01) and student-teacher ratio (14:1 participating vs. 
12:1 not participating; p<0.01). 

Teachers across the BPS program classrooms implemented two 
curricula during the school year: Opening the World of Learning 
(OWL; Schickedanz, Dickinson, & Charlotte-Mecklenberg Schools, 
2005), a language and literacy curriculum; and Building Blocks 
(Clements & Sarama, 2007a), a mathematics curriculum. Both 
curricula have shown positive effects on children’s outcomes 
(e.g., Ashe, Reed, Dickinson, Morse, & Wilson, 2009; Clements & 
Sarama, 2007b; Clements, Sarama, Spitler, Lange, & Wolfe, 2011). 
All prekindergarten teachers received curriculum-specific training 
before the start of the school year, specifically two days of training 
in Building Blocks and five days in OWL. In the previous school year, 
all teachers received in-classroom bi-weekly to monthly coaching 
(Weiland & Yoshikawa, 2013). During the 2009-2010 school year, 
prekindergarten teachers in schools undergoing National Associ- 
ation for the Education of Young Children Accreditation and new 
teachers received bi-weekly on-site support from an experienced 
early childhood coach trained in both curricula. 


2.2. Procedures 


Children were assessed during one-on-one pull-out sessions 
with trained assessors in the child’s school in fall 2009 and spring 
2010. In addition to demonstrating good rapport and management 
skills during training sessions and real testing situations, asses- 
sors had to show reliability on the battery of tests administered 
in the study. Study children were tested in English unless they 
spoke Spanish as a home language and failed practice items on the 
vocabulary assessment (which was the first assessment in the test- 
ing battery). Ultimately, six children in the present sample were 
assessed on their executive function in Spanish (sensitivity analyses 
in which we dropped all non-English home language children and 
refit our primary child-level models show our results were robust 
to child home language; results available upon request). 

To assess classroom quality, trained independent assessors 
observed in study classrooms in spring 2010 for 4-5 h. These obser- 
vations took place on days when general classroom routines could 
be observed (e.g., not on field trip days). Each assessor had to 
demonstrate at least 85% exact agreement with a master coder on 


each measure to ensure reliability. The percent exact agreement 
ranged from 85% to 95%. 

BPS developed the school climate measure based on research 
on three general areas related to student learning: school’s aca- 
demic emphasis, teacher variables, and principal leadership (BPS 
Office of Data and Accountability, n.d.). The district-wide school cli- 
mate surveys were administered in the spring of 2010 to students 
(grades 3-11), parents (grades P-12), and teachers (grades P-12). 
Surveys were distributed in March and returned at the beginning 
of May in the same year. All responses were anonymous. Teach- 
ers and parents had the option to take the survey online or via 
paper and pencil. All students completed the survey in school using 
a Scantron form. Given the low rates of response from parents 
(13.5%), which is consistent with the literature on parent response 
rates of school climate surveys (Austin, 2011), we used only the 
student and teacher survey responses in the present study. Stu- 
dent and teacher response rates were not available at the school 
level. However, respondent data across all BPS schools show that 
approximately 2246 teachers and 23,301 students responded to 
the climate survey, representing 53.3% of P-12 teachers and 57.5% 
of students in grades 3-11. Our district-level teacher response rate 
is slightly lower than other districts testing the influence of school- 
level climate on student achievement outcomes (range: 63-100%; 
Kraft et al., 2016; Nathanson et al., 2013; Scott et al., 2014). Reports 
from students and teachers were aggregated to create school-level 
scores, which are discussed below. 


2.3. Measures 


2.3.1. Classroom-level outcomes 

To assess classroom quality, we used the emotional support, 
classroom organization, and instructional quality domain scores of 
the Classroom Assessment Scoring System (CLASS; Pianta, La Paro, 
& Hamre, 2008). The CLASS is a widely used measure with good psy- 
chometric properties. We chose it because of its focus on the aspects 
of quality related to teacher practice and teacher-child interac- 
tions; we theorized these would more likely be influenced by school 
climate than the more structural elements captured by some other 
commonly used measures (i.e., ECERS). The reliability of the emo- 
tional support, classroom organization, and instructional support 
subscales are 0.68, 0.78, and 0.93, respectively (Hamre et al., 2013). 


2.3.2. Child-level outcomes 
We measured children’s receptive vocabulary using the Peabody 
Picture Vocabulary Test III (PPVT-III; Dunn & Dunn, 1997). The PPVT- 
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III is a widely used measure of receptive vocabulary that has been 
validated on a nationally representative sample of children and 
has excellent test-retest reliability. Children were shown a series 
of four pictures and were asked to choose the picture that corre- 
sponds to the stimulus word. For our analyses, we used the raw 
score for the PPVT-III, which is consistent with evaluations of pub- 
licly funded prekindergarten programs (e.g., Weiland & Yoshikawa, 
2013; Wong, Cook, Barnett, & Jung, 2008). 

We measured EF using a composite of three principal dimen- 
sions: working memory, cognitive inhibitory control, and attention 
shifting. We used the Forward Digit Span and Backward Digit Span 
(FDS and BDS, respectively; Gathercole & Pickering, 2000) tasks to 
measure working memory. During these tasks, children repeated a 
string of numbers in the exact order spoken by the assessor (in FDS) 
or in reverse order (in BDS). The tasks measure different dimensions 
of working memory. BDS measures the central executive compo- 
nent, while FDS measures the phonological loop. BDS demonstrates 
high correlations with other working memory tasks—both ver- 
bal and spatial (Alloway, Gathercole, Kirkwood, & Elliot, 2008; 
Carlson, Moses, & Breton, 2002), as well as good test-retest reliabil- 
ity (r=0.73; Lipsey et al., 2017). FDS demonstrates high correlations 
with BDS and other EF tasks, and has good test-retest reliabil- 
ity in prekindergarten children (r=0.80; Muller, Kerns, & Konkin, 
2012). 

Although the BDS task has been used with children as young 
as four (Alloway, 2007; Gathercole, Brown, & Pickering, 2003), this 
task has illustrated floor effects in prekindergarten samples (Bull, 
Epsy, & Wiebe, 2008; Carlson, 2005). In the current sample, we 
examined the potential for this, and found that only 13% of our 
sample could either not pass the practice items or had missing data 
on this task, and a majority of children received a score that aligns 
with prior mean scores in prekindergarten samples (~M=1.30; 
Fuhs, Nesbitt, Farran, & Dong, 2014; Hamre, Pianta, Hatfield, & Jamil, 
2014). We also did not see any performance differences in passing 
the practice items by home language status, which suggests that 
number word knowledge or language difficulty was not of concern 
in this task. 

To assess cognitive inhibitory control, we used the Pencil Tap 
(Diamond & Taylor, 1996). The task calls for children to tap once if 
the assessor taps twice and to tap twice ifthe assessor taps once. The 
child’s score corresponds to the number of items he/she got correct 
out of 16 possible. The Pencil Tap has demonstrated convergent and 
predictive validity (Blair & Razza, 2007; Lipsey et al., 2017; Rhoades, 
Greenberg, Lanza, & Blair, 2011) as well as test-retest reliability in 
prekindergarten samples (e.g., r=0.80; Lipsey et al., 2017). 

To measure attention shifting in the fall only, we used the 
Dimensional Change Card Sort (DCCS; Frye, Zelazo, & Palfai, 1995) 
task. For this assessment children were shown a series of cards and 
asked to sort the cards according to one dimension (e.g., shape, 
color). Then, they were asked to sort the cards based on another 
dimension. The DCCS total score was the number of trials out 
of 10 in which the child could shift attention correctly from one 
criterion to the next. The DCCS is “the most widely used executive 
function task in early childhood” (Willoughby & Blair, 2016, p. 
92) and demonstrates convergent validity (Carlson, Faja, & Beck, 
2015). It has also been included as a standardized measure in the 
NIH Toolbox where it has demonstrated construct validity (Zelazo 
et al., 2013). 

We chose these specific EF measures because they are predic- 
tive of academic achievement (Bierman, Nix, Greenberg, Blair, & 
Domitrovich, 2008; Fuhs et al., 2014; Weiland et al., 2014). Arecent 
analysis has also compared BDS, Pencil Tap, and DCCS to each other 
and nine other EF measures, and judged these measures on their 
ability to predict achievement, respond to developmental change, 
and relate to teacher-rated classroom behavior (Lipsey et al., 2017). 
BDS, Pencil Tap, and DCCS (and three other tasks) were deemed as 


some of the most educationally relevant measures to use during 
the prekindergarten period. 

We created a single measure of children’s EF skills in the fall 
and spring by taking a unit-weighted average of their FDS, BDS, 
Pencil Tap, and DCCS scores at each time point.! To standardize 
these scores, we divided each child’s test score by the maximum 
score observed for each test and averaged the standardized scores 
separately for the fall and spring outcomes. The unidimensional fac- 
tor can be interpreted as higher scores representing better overall 
executive functioning skills. 

We chose to form a composite EF measure for several reasons. 
Previous evidence — including work on our sample - suggests that 
in the prekindergarten period, EF is comprised of a single latent 
factor (e.g., Barata, 2011; Fuhs & Day, 2011; Weiland et al., 2014). 
Further, other research has combined the specific measures we 
used in our study and found that including a composite score 
versus including separate EF components in their analyses did not 
change their results (Lipsey et al., 2017). Also, one study found 
that the test-retest reliability and developmental trajectories for 
EF between ages 3-5 were different depending on whether EF was 
measured as a composite or as individual test scores (Willoughby 
& Blair, 2016); they conclude that across-time stability and sensi- 
tivity to developmental changes in EF was more accurate when a 
composite score was used. 


2.3.3. Child- and classroom-level sociodemographic covariates 

We used district administrative records to obtain information 
about children’s race/ethnicity, school mobility, gender, home lan- 
guage, free- and reduced-lunch status, special-needs status, and 
school attendance zone. We constructed a series of dichotomous 
indicators to represent child race/ethnicity (i.e., Asian, Black, His- 
panic, White, Other), home language (i.e., English, Spanish, Other), 
attendance zone (i.e., north, east, west), free-reduced lunch eligi- 
bility, male, and special needs, each coded as one when the child 
belonged to the requisite group and zero otherwise. White was 
the reference group for child race/ethnicity, English was the refer- 
ence group for home language, and west was the reference group 
for attendance zone. We chose these covariates because they have 
been shown to predict early cognitive and educational outcomes 
in the early childhood literature and thus may confound under- 
standing of associations between school climate and the outcomes 
of interest in the study (e.g., Clements et al., 2011; Wong et al., 
2008). Classroom-level sociodemographic variables were created 
using classroom aggregates of child-level variables. We created 
these variables to account for classroom-level factors that could 
be related to our outcomes of interest. 


2.3.4. Teacher covariates 

For teacher experience, we used a vector of dichotomous indi- 
cators calculated from administrative data that captured whether 
the teacher had between 0-3, 3-5, 5-10 or 10 or more years of total 
teaching experience. Teachers with 0-3 years of teaching experi- 
ence served as the reference group. We used district administrative 
records to create another dichotomous variable that was set to 1 if 
the teacher had a master’s degree and zero otherwise. 


2.3.5. School-level covariates 

We used data from the 2009-2010 academic year retrieved 
from the Massachusetts Department of Elementary and Secondary 
Education’s School and District Reports (n.d.) as our school-level 
covariates. We chose covariates that theory and prior litera- 


1 Previous factor analytic work with this sample (Weiland & Yoshikawa, 2014) 
has shown that configural and metric invariance holds across time for this construct, 
even with DCCS included in the fall only. 
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Fig. 2. Decision process for identifying school climate dimensions. 


ture pointed to as contributors to classroom quality and student 
learning. Specifically, we included covariates for the percent of 
low-income students, the percent of students with disabilities, 
the percent of students who were English language learners, and 
the percent of students who remained in the school from the 
2008-2009 to the 2009-2010 academic year. We also included 
the average school achievement level by including the percent of 
third graders scoring at or proficient in math and English language 
arts. There is some evidence to suggest that academic character- 
istics in the older grades can influence the academic development 
of younger children in the school (e.g., Zhai et al., 2012). Finally, 
we included covariates for the type of school the children attended 
(i.e., prekindergarten to grade 5, prekindergarten to grade 8) using 
a dichotomous indicator set to one if the child attended an elemen- 
tary school and zero if otherwise, the percent of teachers retained 
between the 2008-2009 and 2009-2010 academic years, and the 
average student/teacher ratio of the school. 


2.3.6. School climate 

Dimensions of school climate were all measured using district- 
wide school climate surveys (Boston Public Schools, 2008-2017). 
The teacher and student surveys included a total of 94 items, orga- 
nized by the district into 11 subscales. For the teacher survey, there 
were six BPS-identified school climate dimensions (52 items): work 
collegiality, school leadership, parent and student engagement, 
perceptions of collective efficacy, teacher self-efficacy, and teacher 
influence. For the student survey, there were five BPS-identified 
school climate dimensions (42 items): teacher effectiveness, prin- 
cipal effectiveness, school safety, perceptions of school, and 
enthusiasm for learning. These subscales are consistent with school 
climate surveys used by other large, urban school districts that have 
found relationships between school-level climate and school qual- 
ity (e.g., lower teacher turnover; Kraft et al., 2016; larger percent of 
on-track students and higher test scores; Nathanson et al., 2013). 


For each school climate dimension, we used aggregated school- 
level data from all students and teachers in the school. We made 
this decision because evidence suggests that teachers’ and students’ 
perceptions of school climate are sensitive to different aspects of 
the learning environment, which could be related to how their per- 
ceptions are measured (Thapa et al., 2013). While measures for 
teachers and students often include different sets of questions in 
our dataset, raters are not reporting on disparate topics as the con- 
structs across raters fit within the essential supports framework. 
Including multiple reporters—the recommended approach in the 
school climate literature—could accurately capture the full range 
of school climate dimensions that likely influence children’s learn- 
ing and better support alignment between district-wide measures 
used in practice and established school climate frameworks, like 
the Essential Supports. The goal of our analyses was to combine 
complementary perspectives to gain a more global understanding 
of the experiences of teachers and students who are embedded 
within schools. 

Most items (N= 81 out of 103 items) had the same four-point Lik- 
ert scale (1 =strongly disagree, 2 =disagree, 3=agree, 4=strongly 
agree) that asked respondents to rate agreement with each state- 
ment. Some items (N=12 out of 103 items) had a four-point 
Likert scale that asked respondents to indicate how often s/he 
observed or experienced certain events (1 =never, 2=sometimes, 
3=most times, 4=always), or asked teachers to express how 
much control or influence s/he has over students and classroom 
(1=none/nothing, 2=very little, 3=quite a bit, 4=a great deal 
(N=10 out of 103 items). All items and dimension were scored 
on the same continuum where higher scores demonstrate a more 
positive climate. 

As explained in our literature review, school climate has no 
agreed-upon definition or measurement approach in the field. 
To create our constructs, we adopted a theory- and data-driven 
approach for identifying the school climate dimensions as shown 


S.E. Rochester et al. / Early Childhood Research Quarterly 48 (2019) 84-97 91 
Table 4 
Descriptive statistics and bivariate correlations for the school climate dimensions. 
School climate dimension a Mean (SD) Range 1 2 3 4 5 6 7 8 9 10 
1. Pos emotional climate 0.86 3.11 (0.14) 2.75-3.38 - 
2. Enthusiasm for learning 0.94 3.54 (0.09) 3.35-3.81 0.277 - 
3. Teacher effectiveness 0.93 3.46 (0.11) 3.20-3.64 0.36 0.78" - 
4. Instructional leadership 0.97 1.61 (0.19) 0.99-1.90 0.33 0.19 0.10 - 
5. Teacher influence 0.90 2.71 (0.29) 2.05-3.31 0.28" —0.18 -0.31° 0.54" - 
6. Principal effectiveness 0.96 3.50 (0.17) 3.01-3.78 0.14 0.59" 0.32 0.40" 0.06 - 
7. Work collegiality 0.95 3.12 (0.30) 2.28-3.70 0.29" 0.05 0.16 0.36 0.33" -0.11 - 
8. Teacher self-efficacy 0.86 3.34 (0.19) 2.50-3.68 0.17 0.05 0.13 0.18 0.34" 0.12 0.15 - 
9. Collective efficacy 0.93 3.25 (0.20) 2.72-3.71 0.44" 0.16 0.18 0.39" 0.39" 0.01 0.88" 0.20 - 
10. P and S engagement 0.93 2.71 (0.25) 2.07-3.32 0.32" 0.08 0.02 0.17 0.32" 0.14 0.20 0.34" 0.44" - 
Note. Pos emotional climate = positive emotional climate; Collective efficacy = perceptions of collective efficacy; P and S engagement = Parent and student engagement. 
* p<0.05. 
“ p<0.01. 
“ p<0.001. 


in Fig. 2. First, we screened out items the district included incon- 
sistently across the five years of survey administration (2009-2010 
through 2015-2016), dropping two items (one each from teacher 
and student surveys). We then fit confirmatory factor analyses 
(CFA) models of the factor structures of the 11 BPS-identified 
dimensions of school climate, including student and teacher reports 
together in the models. Fit indices were compared to the recom- 
mended thresholds of CFI and TLI values of at least 0.90, RMSEA 
below 0.05, and SRMR below 0.10 (Browne & Cudeck, 1993; Hu & 
Bentler, 1999). Models meeting at least two of the four indices were 
considered to have adequate fit to the data (particularly as evidence 
suggests that some fit indices, such as RMSEA, do not perform well 
in small samples; Little, Card, Slegers, & Ledford, 2007). 

Indices suggested that the hypothesized model adequately 
fit the data for the following six of 11 school climate dimen- 
sions reported by BPS: work collegiality [CFI=0.93; TLI=0.88; 
RMSEA=0.25; SRMR=0.03], teacher self-efficacy [CFI=1.00; 
TLI=0.99; RMSEA=0.06; SRMR=0.01], teacher influence 
[CFI=1.00; TLI=0.99; RMSEA=0.07; SRMR=0.01], teacher effec- 
tiveness [CFI=0.91; TLI=0.90; RMSEA=0.15; SRMR=0.05], 
principal effectiveness [CFI=0.99; TLI=0.96; RMSEA=0.17; 
SRMR= 0.02], and enthusiasm for learning [CFI=0.92; TLI=0.89; 
RMSEA=0.18; SRMR=0.04]. Three dimensions were from the 
teacher survey (work collegiality, teacher self-efficacy, teacher 
influence) and three from the student survey (teacher effec- 
tiveness, principal effectiveness, enthusiasm for learning). We 
maintained these dimensions for the current study because 
they contained significant overlap with the Essential Supports 
framework and because of their relevance to BPS’s conceptual- 
ization and use of school climate data. For example, the teacher 
influence dimension identified by BPS maps onto the teacher 
influence measure within the framework’s school leadership 
area. Similarly, the BPS-identified dimension termed perceptions 
of collective efficacy is well aligned with Essential Supports’ 
collective responsibility measure within the professional capacity 
support. 

The remaining five dimensions did not have adequate fit and 
we accordingly examined the individual items within each dimen- 
sion for construct validity. We first removed items (N=12) that 
we deemed inappropriate for third graders, the youngest grade 
of reporters (e.g., for the item, “I brag about my school to friends 
who do not attend this school”, it was unclear which aspects of 
their schooling environment children were judging when deter- 
mining whether to brag about their school). Next, we dropped items 
(N= 22) that did not cleanly fit on any one dimension (e.g., “Your 
students come to school ready to learn”. Here, teachers’ perceptions 
about whether their students are ready to learn could be influenced 
by children’s day-to-day experiences, such as completing home- 


work when assigned, or their views about/collegiality with their 
students’ previous-grade teacher). The screening process resulted 
in a total of 34 items being dropped. 

We then examined the remaining items (N=27) and subscales 
for conceptual coherence using recent school climate research 
(e.g., Cohen et al., 2009; Thapa et al., 2013) and the Essential 
Supports framework (Bryk et al., 2010), which led to combin- 
ing the original BPS dimensions of school safety and perceptions 
of school to create a new dimension of positive emotional cli- 
mate. School leadership was renamed instructional leadership to 
more accurately reflect the items. Finally, we refit the CFAs with 
the remaining items and subscales, applying the same adequate 
fit criteria. The iterative process resulted in four school climate 
dimensions: parent and student engagement [CFI = 0.96; TLI = 0.92; 
RMSEA= 0.25; SRMR=0.02], instructional leadership [CFI=0.97; 
TLI=0.94; RMSEA=0.24; SRMR=0.01], perceptions of collective 
efficacy [CFI=0.99; TLI=0.99; RMSEA=0.07; SRMR=0.01], and 
positive emotional climate [CFI=0.98; TLI=0.93; RMSEA=0.13; 
SRMR=0.04]. A complete list of school climate dimensions 
and items can be found in Supplementary Appendices A and 
B. 

As shown in the last column of Table 1, our analyses resulted 
in 10 dimensions of school climate, six dimensions originally iden- 
tified by BPS and four researcher-constructed dimensions based 
on the Bryk et al. (2010) framework. We calculated a score for 
each dimension by taking a unit-weighted average of the items 
for that dimension. Cronbach’s alpha for each dimension ranged 
from 0.86 to 0.97 (see Table 4). The final 10 school climate 
dimensions used as predictors of observed prekindergarten class- 
room quality and children’s receptive vocabulary and EF skills 
across the prekindergarten year include the following: work col- 
legiality, instructional leadership, parent and student engagement, 
perceptions of collective efficacy, teacher self-efficacy, teacher 
influence, teacher effectiveness, principal effectiveness, positive 
emotional climate, and enthusiasm for learning. See Supplemen- 
tary Appendix C for a detailed description of each school climate 
dimension. 


2.4. Data analytic approach 


To address the first research question — whether higher school 
climate scores as reported by teachers and older children (grades 
3-8) were associated with higher observed prekindergarten class- 
room quality - we fit a series of hierarchical multilevel regression 
models with random intercepts for schools. We first regressed 
classroom quality on each dimension of school climate. We then 
added our other covariates to the model in turn in conceptual blocks 
(i.e., first, classroom-level covariates; second, teacher experience 
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Table 5 
Associations between school climate and classroom quality. 


Aspect of classroom quality 


Emotional support 


Classroom organization Instructional support 


Predictor M1 M2 M3 M4 
School climate dimension 
Positive emotional climate 2.14” 2.06" 1.75 2.32" 
(0.63 (0.76) 0.76) 0.89) 
Teacher effectiveness 1.59 1.40 1.79° 2.29" 
(0.84 (0.83) 0.85) 0.96) 
Enthusiasm for learning 1.39 0.97 1.01 1.65 
(1.08 (0.99) 1.08) 1.40) 
Instructional leadership 0.03 0.08 0.17 —0.20 
(0.57 (0.53) 0.55) 0.63) 
Teacher influence —0.04 —0.25 —0.36 —0.50 
(0.36 (0.36) 0.39) 0.43) 
Principal effectiveness 0.16 0.26 —0.02 —0.48 
(0.55 (0.51) 0.55) 0.86) 
Work environment collegiality 0.17 —0.25 —0.25 —0.27 
(0.32 (0.33) 0.35) 0.44) 
Teacher self-efficacy 0.96 0.61 —0.13 —0.10 
(0.58 (0.58) 0.72) 0.77) 
Perceptions of collective efficacy 0.32 —0.70 —0.80 —1.14 
(0.48 (0.55) 0.56) 0.73) 
Parent and student engagement 0.98" 0.20 0.25 0.08 
(0.39 (0.51) 0.51) 0.73) 
Classroom sociodemographic characteristics x Xx xX 
Teacher experience and classroom quality x x 
School characteristics xX 


M1 M2 M3 M4 M1 M2 M3 M4 
164 89159 §=1.45 2.12 1.61 1.35 0.92 0.93 
0.77) (0.79) (097) (1.08) (092) (1.18) (132) (1.51 
1.86 166 216 288 1.01 0.76 1.25 2.29 
0.97) (084) (1.02) (1.16) (1.18) (1.24) (140) (1.58 
1.02 0.63 0.87 1.97 0.46 015 0.25 1.88 
1.24) (1.08) (1.34) (1.77) (145) (148) (1.68) = (2.12 
0.18 0.03 0.04 047 -086 -112 -1.03 —-1.98° 
0.63) (057) (069) (080) (073) (0.76) (084) (087 
0.20 -027 -044 -0.72 -032 -057 -0.73 -1.19 
0.40) (038) (048) (055) (046) (052) (059) (0.62 
027 -015 -024 -020 -076 -110 -115 —-1.98 
0.63) (0.55) (069) (1.05) (071) (074) (084) (1.24 
0.28 0.12 -005 -018 0.10 0.62 -0.50 —0.88 
0.37) (036) (044) (0.55) (042) (049) (0.54) (0.66 
0.65 0.33 0.15 -0.08 0.12 ~039 0.90 —-0.83 
0.64) (0.60) (0.79) (0.82) (078) (085) (1.03) (1.07 
0.17 0.77. -079 -150 0.31 1.08  -097 —1.46 
0.55) (0.58) (0.70) (089) (063) (080) (087) (1.08 
0.92 041 0.46 0.35 0.77 0.01 -012 -0.15 
0.44) (052) (058) (0.79) (054) (0.74) (0.78) (1.04 
x x xX x x xX 
xX X x x 
x x 


Note. Classroom models control for classroom sociodemographic characteristics (gender, mobility, free-reduced-lunch eligibility, special needs status, race/ethnicity, home 
language, and attendance zone), teacher background (years of experience, teacher education), and school characteristics (student/teacher ratio, percent of students who 
remained in the school/district from the 2008-2009 to 2009-2010 academic year, percent of low income students, percent of students with disabilities, percent of teachers 
retained, percent of third-graders at or proficient on standardized English language arts exam, the percent of third-graders at or proficient on standardized math exam, and 
whether the school was PK - 5). The “Xs represents the group of variables that was introduced to model in conceptual blocks. This table shows results from 30 different series 
of random effects multilevel regression models with random effects for schools, where CLASS emotional support, CLASS classroom organization and CLASS instructional 
support were regressed onto each separate school climate dimension. Robust standard errors in parentheses. 


~ p<0.10. 
*“ p<0.05. 
“ p<0.01. 


and education; and finally, school-level covariates). We examined 
the stability of point estimates and standard errors for the key ques- 
tion predictors — dimensions of school climate - across this set of 
four models per outcome. 

We followed the same conceptual block model-building proce- 
dure in examining the second research question — whether school 
climate was associated with gains in receptive vocabulary and EF 
- and fit a series of hierarchical multilevel regression models with 
random intercepts for schools. In our first conceptual block, how- 
ever, we regressed end-of-year receptive vocabulary and EF on each 
dimension of school climate and the associated beginning-of-year 
child outcome. We then added our covariates in conceptual blocks 
(i.e., first, child-level sociodemographic characteristics; second, 
teacher experience and classroom quality; and finally, school-level 
covariates). 

There was a small amount of missing data at the child and class- 
room level, ranging from 2% to 10% (see the notes in Tables 2 and 3 
for more details). We used multiple imputation (50 datasets) to 
account for missing child-level pretest and classroom-level data 
with all analytical variables used in the imputation model (Graham, 
2009). Findings were stable across models that used imputed and 
non-imputed data (see Supplementary Appendix D). All regression 
analyses were conducted in Stata 13. 


3. Results 
3.1. Descriptive statistics: School climate predictors 


We first calculated descriptive statistics on the school-level 
climate dimensions. On average, children were in schools that 
had a positive climate. For example, teachers were rated to have 
high rates of effectiveness (M=3.46; SD=0.11; Range = 3.20-3.64) 
and students reported themselves as being engaged in their own 
learning (M=3.54; SD=0.09; Range = 3.35-3.81). However, schools 
had relatively low instructional leadership (M=1.61; SD=0.19; 
Range = 0.99-1.90), indicating that on average, teachers felt that 
principals had low expectations for and involvement in teaching 
and learning. As shown in Table 4, correlations between dimensions 
of school climate ranged from 0.29 to 0.88. When we examined the 
correlations in light of the Essential Supports framework, we found 
some statistically significant correlations among school climate 
dimensions within the same overall support area. For example, the 
following three school climate dimensions addressed the student- 
centered learning climate support: positive emotional climate, 
teacher effectiveness, and student enthusiasm for learning. We 
found generally moderate to strong statistically significant corre- 
lations among these climate dimensions (range of 0.27-0.78). 
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Table 6 
Associations between school climate and prekindergarten gains. 


Prekindergarten outcome 


Receptive vocabulary 


Executive function 


Predictor M1 M2 M3 M4 M1 M2 M3 M4 
School climate dimension 
Positi F Ili 4.69 —3.76 —0.86 —1.76 —0.01 —0.06 —0.06 —0.16 
osiveemotional cumate (7.31) (7.73) (8.90) (12.01) (0.07) (0.07 (0.08) (0.11) 
Teacher effectiveness 3.51 3.71 12.65 6.04 0.11 0.12 0.09 0.03 
(9.21) (9.08) (10.01) (11.77) (0.09) (0.09 (0.10) (0.11) 
Enthusiasm for learning 8.37 9.77 19.12 7.32 0.12 0.13 0.11 0.01 
(11.46) (11.14) (12.39) (15.12) (0.11) (0.11 (0.12) (0.14) 
Instructional leadership 115. —0.56 —0.08 —3.92 0.00 0.01 —0.00 —0.05 
5.51) (5.49 (6.00) (7.13 0.05) (0.05 (0.06) 0.07) 
Teacher influence 0.40 —1.07 —2.19 —0.54 —0.04 —0.05 —0.05 —0.05 
3.48) (3.57 (3.78) (4.21 0.03) (0.03 (0.04) 0.04) 
Principal effectiveness 1.88 4.36 4.87 —2.50 0.01 0.05 0.05 0.01 
5.19) (5.18 (5.92) (8.28 0.05) (0.05 (0.06) 0.08) 
Work es, icotlesialit 2.79 —0.77 —0.95 —0.62 0.04 0.01 —0.01 —0.01 
OC CHV FOnMEnT cOucela. 3.34) (3.43 (3.62) (4.47 0.03) (0.03 (0.03) 0.05) 
Teacher self-efficacy 5.56 4.17 1.81 2.22 —0.00 0.01 0.02 0.03 
6.68) (6.65 (8.01) (8.14 0.07) (0.07 (0.08) 0.08) 
Parcestions-of collective-athcac 7.08 —0.76 —1.77 —0.90 0.05 —0.03 —0.04 —0.07 
recee : y 5.00) (5.54 (5.78) (7.85 0.05) (0.05 (0.06) 0.08) 
Parentand studentiengasement 5.38 —0.18 0.96 415 0.05 0.02 0.02 0.04 
a8 (4.16) (4.45 (4.93) (7.06 (0.04) (0.04 (0.05) (0.07) 
Child sociodemographic characteristics x xX x xX X 
Teacher experience and classroom quality x X X 
School characteristics xX Xx 


Note. Student models control for child sociodemographic characteristics (gender, mobility, free-reduced-lunch eligibility, special needs status, race/ethnicity, home language, 
and attendance zone), teacher experience and classroom quality (years of experience, teacher education, emotional support, classroom organization, instructional support), 
and school characteristics (student/teacher ratio, percent of students who remained in the school/district from the 2008-2009 to 2009-2010 academic year, percent of low 
income students, percent of students with disabilities, percent of teachers retained, percent of third-graders at or proficient on standardized English language arts exam, 
the percent of third-graders at or proficient on standardized math exam, and whether the school was PK-5). The “X”s represents the group of variables that was introduced 
to model in conceptual blocks. This table shows results from 20 different random effects hierarchical regression models with random effects for schools, where receptive 
vocabulary and executive function were regressed onto each separate school climate dimension. Robust standard errors in parentheses. 


Before fitting our regression models, we calculated the propor- 
tion of total outcome variation that lies “between” schools, also 
called the school-level intraclass correlation (ICC) for our classroom 
quality and prekindergarten vocabulary and EF outcomes (Singer & 
Willett, 2003). The ICCs for the domains of classroom quality in null 
models were 0.17, 0.19, and 0.31 for emotional support, classroom 
organization, and instructional support, respectively. The school- 
level ICCs in null models for receptive vocabulary and EF were 0.02 
and 0.10, respectively. All ICCs suggest that the majority of the 
outcome variation lies within schools, rather than between them. 


3.2. Regression analyses 


For research question one - the relationship between school 
climate and observed classroom quality in prekindergarten —- we 
found statistically significant associations between two dimen- 
sions of school climate and classroom quality. Table 5 displays 
these results by classroom quality dimension. In each classroom 
quality panel, Model 1 shows results of including only the school 
climate indicator of interest. Model 2 included both the school cli- 
mate dimension and classroom sociodemographic characteristics. 
Model 3 included the variables from Model 2 and teacher experi- 
ence and quality. Model 4 included the variables in Model 3 and 
school characteristics. 

We focus on interpreting results from the full model given that 
it likely best controls for potential school/classroom-level endo- 
geneity and because of the relative stability of point estimates and 
standard errors across all four conceptual blocks. As shown in col- 


umn 5 of Table 5, having a positive emotional school climate was 
statistically significant and positively predictive of prekindergarten 
classroom emotional support (8 =2.32, p=0.009), controlling for 
teacher, classroom, and school characteristics. Using parameter 
estimates from Model 4, this association translated into moderate 
standardized coefficient of 0.47, using Cohen’s d (e.g., multiplying 
the predictor’s coefficient by the standard deviation of the predic- 
tor and dividing by the standard deviation of the outcome; NICHD 
ECCRN & Duncan, 2003). We also found that teacher effective- 
ness was a Statistically significant, positive predictor of emotional 
support (8 =2.29, p=0.017) and classroom organizational support 
(B =2.88, p=0.013) controlling for teacher, classroom, and school 
characteristics (see columns 5 and 9 of Table 5). These associations 
translated to moderate standardized coefficients of 0.37 and 0.45, 
respectively. No other school climate models were predictive of 
observed prekindergarten classroom quality across all four models. 

For research question two, we followed the same model- 
building strategy. We found no evidence of an association between 
school climate and children’s gains in receptive vocabulary (see 
columns 2-5, Table 6) or in children’s EF skills (see columns 6-9, 
Table 6). There was no consistency in the direction of associations; 
for example, for receptive vocabulary across the most controlled 
model (M4), four associations were positive in direction and six 
were negative. 

Finally, to ensure that our null findings for EF were not due to 
our decision to form a composite EF score (provided past work on 
differential effects of classroom processes on EF by specific EF com- 
ponent [Hamre et al.,2014]), we refit our primary models with each 
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of the EF tasks as outcomes. In addition, to ensure that our null find- 
ings were unrelated to time between fall and spring testing, we 
also refit our child-level models controlling for days between test- 
ing dates. Our results were not sensitive to these analytic decisions 
(see Supplementary Appendix F). 


4. Discussion 


Our findings extend the current literature by examining the role 
of school climate during prekindergarten, a developmental period 
few previous studies in this literature have considered. We used 
current domains of school climate as described in the research 
literature (Berkowitz et al., 2017; Thapa et al., 2013) and the Essen- 
tial Supports framework, a comprehensive and empirically-tested 
model of school supports in elementary school settings (Bryk et al., 
2010), to establish the reliability and validity of a district-wide 
teacher and student survey. At the classroom level, we found associ- 
ations between classroom emotional support and two dimensions 
of school climate, positive emotional climate and teacher effective- 
ness. We also found an association between classroom organization 
and teacher effectiveness. These associations were robust across 
model specifications and were small to moderate in magni- 
tude (Cohen’s d=0.35-0.55). School climate was not related to 
prekindergarten children’s receptive vocabulary and EF gains. 

Our classroom-level findings suggest that prekindergarten 
classrooms in schools that older children consider safe and aca- 
demically supportive tend to have teachers who maintain positive 
relationships with their students and teachers that can successfully 
manage day-to-day classroom tasks (e.g., behavior management, 
instructional learning formats). Notably, both of these school cli- 
mate dimensions fit within the student-centered learning climate 
essential in the Essential Supports framework (Bryk et al., 2010). 
While these findings are promising for the use of school climate 
measures, the drivers behind these relations are unclear. It could 
be that prekindergarten teachers have more teaching time due 
to fewer disruptions to school and classroom routines in schools 
where older students feel safe and perceive as orderly. Or per- 
haps more emotionally supportive prekindergarten classrooms are 
indicative of the overall support that schools provide, which is con- 
sistent with older students’ perception that school teachers create 
a supportive academic environment. In other words, prekinder- 
garten teachers may be embodying the schools’ positive approach 
to learning in their interactions with the children in their classroom. 
Because our investigation is correlational, it could also be that a 
more positive emotional climate is reflective of classrooms that are 
more emotionally supportive. Or perhaps stronger prekindergarten 
teachers select into schools with more supportive climates. 

It is important to note that for 8 out of 10 school climate dimen- 
sions, we found no relation between broader school climate and 
classroom quality across our four conceptual blocks. Our mix of sta- 
tistically significant associations and null findings between some 
measures of school climate and classroom quality findings fit with 
the extant literature, in which findings have been mixed as well. 
While relational and organizational climate had small associations 
with global measures of classroom quality (Dennis & O’Connor, 
2013), these relational school climate dimensions were not pre- 
dictive of specific domains of classroom quality like emotional 
support (Zinsser & Curby, 2014). Perhaps more proximal school 
climate characteristics that influence children’s day-to-day inter- 
actions, like teacher support and students’ physical and emotional 
safety, are more important for specific aspects of the prekinder- 
garten classroom environment. 

At the child level, we did not find evidence to support a link 
between school climate and children’s gains in receptive vocabu- 
lary and EF. These findings were somewhat surprising given that 
several studies have found that school-level adult support is pre- 


dictive of older children’s academic outcomes (e.g., Hopson & Lee, 
2011; Lucio, Hunt, & Bornovalova, 2012; Stewart, 2008). Also, from 
a theoretical perspective, a more positive school climate could have 
increased teachers’ expectations for students, leading to higher- 
level instruction including use of more sophisticated vocabulary. 
For EF, school climate might have been expected to influence chil- 
dren’s EF gains as prekindergarten children attending schools with 
more positive climates might have had more structure and rou- 
tine in their schooling environment, which may have created more 
opportunity for children to effectively practice working memory 
and be supported along the way. 

Drawing on theories of bioecological systems that posit chil- 
dren are nested within multiple contexts (Bronfenbrenner, 1977; 
Bronfenbrenner, & Morris, 1998), it could be that prekindergarten 
teachers serve as a buffer to the broader schooling context, such 
that children’s daily classroom experiences (e.g., teacher—child and 
peer-to-peer interactions) are more salient to the development 
of their foundational vocabulary and self-regulatory skills than 
school-wide occurrences. Another explanation for our null associa- 
tions with children’s cognitive gains could be that prekindergarten 
children spend less time outside of their classrooms than older chil- 
dren and therefore are less likely to be directly affected by their 
school’s broader climate. We are not aware of studies that identify 
how prekindergarten children in public schools spend their time 
versus children in the older grades in their same school. Providing 
such evidence in future studies would help to further unpack how 
broader school climate may influence prekindergarten children’s 
learning and development. 

Notably also, the prior literature on older children did not exam- 
ine vocabulary or executive function. It could be the case that 
school climate affects other child outcomes in prekindergarten, 
just not the ones measured in our study. For example, children’s 
behavior can be particularly contextually dependent; more posi- 
tive school climate could be associated with reductions in children’s 
problem behaviors across the prekindergarten year. Also regarding 
measurement, we view it as unlikely that our null findings for EF 
are due to our choice of EF tasks given past work on how class- 
room processes predict children’s gains on the Pencil Tap (Williford, 
Whittaker, Vitiello, & Downer, 2013) and BDS (Hamre et al., 2014) 
EF tasks in prekindergarten and given the comparatively good per- 
formance of our tasks versus others in a recent prekindergarten 
study (Lipsey et al., 2017). 

There are a number of limitations to this study. First, we were 
unable to identify the grade of student and teacher raters within 
our school climate data. As a result, it is unclear if there are dif- 
ferences (or similarities) in school climate perceptions between 
older and younger elementary teachers and students. Due to our 
sample size, we were unable to examine measurement invariance 
across teachers and students. Future research should also collect 
rater demographic data (e.g., grade, age) and examine the poten- 
tial influence of additional school-level characteristics on perceived 
school climate. Second, there was not much variation in climate 
dimension scores across the district surveys at the school-level 
(e.g., enthusiasm for learning M=3.54, SD=0.09), which limited 
our ability to use them as predictors of classroom quality and 
children’s academic outcomes and to use multilevel measurement 
models to operationalize constructs. Future studies should employ 
multilevel measurement models and explore whether classroom 
quality mediates the relationship between school climate and 
prekindergarten children’s academic outcomes. Third, our inves- 
tigation included one school district in which all prekindergarten 
programs were within urban public schools which limits the exter- 
nal validity of our findings. 

Fourth, we fit 60 regression models in our analyses (four 
models for each school climate predictor across five outcomes) 
that included a relatively small sample of prekindergarten chil- 
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dren, which could have led to some null and spurious results. 
Given the exploratory nature of our study and consistent with 
current guidelines (Schochet, 2009), we did not adjust for multiple 
comparisons. We emphasized results that were stable across 
models for a given predictor and outcome (four total) to help 
limit attention to spurious results. Fifth, our exploratory study is 
correlational and not causal in nature. Sixth, we lacked data on 
how integrated prekindergarten classrooms in our sample were in 
the broader school (e.g., close to K and above classrooms, versus 
in the basement or in an annex). Such data would be useful in 
future studies in understanding how the broader school climate 
does or does not affect prekindergarten classrooms processes. 
Also, we do not consider the relationship between school climate 
and home factors (e.g., home emotional climate). For example, it 
is unclear whether school climate matters most for children for 
whom there is a bigger contrast with their home climate. Future 
studies should consider the influence of school climate dimensions 
on other child outcomes that may be particularly relevant for the 
prekindergarten period, such as the aforementioned home factors 
and children’s social-emotional skills. 

Finally, the school climate measures used in our study do not 
explicitly take the full prekindergarten context into account. For 
example, many prekindergarten programs place a large emphasis 
within programmatic activities to support parents (e.g., support- 
ing parents in the transition to kindergarten), which is not often 
reflected in existing school climate measures. Research has only 
recently begun to consider the role that school climate may have 
on early childhood settings. Drawing from the Essential Supports 
framework (Bryk et al.,2010),ateam of scholars from the University 
of Chicago has started to examine how climate-like characteris- 
tics of school- and center-based programs (e.g., socio-emotional 
instruction, teacher outreach and collaboration with parents) as 
reported by teachers and parents may support program-level qual- 
ity and instructional quality in prekindergarten classrooms (Ehrlich 
et al., 2016; Ehrlich, Pacchiano, Stein, Wagner, 2018; Ehrlich, 
Pacchiano, Stein, Wagner, Park et al., 2018). Ehrlich, Pacchiano, 
Stein, & Wagner (2018) and Ehrlich, Pacchiano, Stein, Wagner, Park 
et al. (2018) found some evidence - like the findings from the cur- 
rent study — that prekindergarten programs with more supportive 
environments (e.g., positive learning climate, teacher safety) were 
related to site-level characteristics, such as increased student atten- 
dance. As measures become available, districts serving younger 
children should also consider integrating climate-like measures 
into their existing survey efforts. Future work should also continue 
to explore the role of school and organizational climate in early 
childhood settings on a broader range of site-level characteristics 
(e.g., teacher turnover). 

Despite these limitations, our study contributes to the cur- 
rent policy and practice discussion nationally and internationally 
regarding the use of school climate measures in school improve- 
ment and accountability efforts. Our classroom-level findings 
suggest that school climate may have implications for the quality 
of prekindergarten students’ experiences and that prekindergarten 
should not be left out of ESSA school climate discussions. If our 
findings replicate in other settings, local districts should, for exam- 
ple, consider taking school climate into account if they have 
choice in how to expand their prekindergarten programs. Keeping 
prekindergarten children in mind when making school improve- 
ment decisions may ensure that the school’s youngest learners ben- 
efit from a broader positive and supportive learning environment. 


Appendix A. Supplementary data 


Supplementary material related to this article can be found, in 
the online version, at doi:https://doi.org/10.1016/j.ecresq.2019.02. 
008. 
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